TOJET: The Turkish Online Journal of Educational Technology - October 2012, volume 11 Issue 4 


EVALUATING THE IMPACTS OF ICT USE: A MULTI-LEVEL ANALYSIS WITH 
HIERARCHICAL LINEAR MODELING 

Hae-Deok Song 
Department of Education 
Chung-Ang University 
Seoul, South Korea 
hsong@cau.ac.kr 

Taehoon Kang 
Department of Education 
Sungshin Women’s University 
Seoul, Korea 

taehoonkang@sungshin.ac.kr 


ABSTRACT 

The purpose of this study is to evaluate the impacts of ICT use on achievements by considering not only ICT 
use, but also the process and background variables that influence ICT use at both the student- and school-level. 
This study was conducted using data from the 2010 Survey of Seoul Education Longitudinal Research. A 
Hierarchical Linear Modeling (HLM) analysis was employed to control the student and school characteristics. 
The study results show that the use of ICT explained a significant portion in the overall variance in mathematic 
achievement at the elementary school level, at the middle school level, and at the high school level, respectively. 
Information communication and transactions, a component of ICT literacy, has negative impacts on mathematic 
achievement, and ICT self-efficacy is more likely to result in academic achievement than other background and 
processing variables. 

INTRODUCTION 

Infonnation and Communication Technology (ICT) is widely used in education for collection, management, and 
analysis. ICT in education includes a variety of tools, such as computers, CD-Roms, projection TVs, word 
processors, image graphic software, email, and Internet-based communication technology. ICT use can influence 
teaching and learning styles by changing the emphasis from a teacher-centered to a learner-centered style and 
provides opportunities to improve information-reasoning skills, communication skills, higher thinking skills, 
creativity, and problem solving (Shaikh & Khoja, 2011; Yusuf & Afolabi, 2010). 

Despite the potential advantages of ICT use, the impacts of ICT on learning outcomes have not shown consistent 
results (Aristovnik, 2012; Shaikh, & Khoja, 2011). One main reason for this may be explained by a 
methodological limitation (Cox & Marshall, 2007). Previous studies did not differentiate the pure effects of ICT 
use from the effects of other variables. A variety of factors could affect the relationship between ICT use and 
achievement; analyzing the impacts of ICT use requires studying the single contribution of ICT use in 
classrooms. However, previous studies were limited in that they did not differentiate the impacts of the ICT use 
variables from other variables related to ICT use processes or backgrounds. Given that ICT is used for different 
purposes, it is necessary to examine which variables are significant. Furthermore, previous studies did not 
consider the difference levels among ICT-related variables. Variables that directly or indirectly affect ICT use 
can be categorized into two levels: student-level and school level. Variables at the student-level include students’ 
capabilities to use ICT, gender, parents’ education levels, and private tutoring expenditures. On the other hand, 
variables at the school-level include teachers' education levels, attitudes toward education reform, and principal’s 
leadership. To clarify the effects of ICT use, it is necessary to examine how these variables affect achievement 
and which variables are significant. Finally, to further examine the effects of ICT use at school, it is also 
necessary to examine how ICT is being used at different grade levels, namely elementary, middle, and high 
school. 

The hierarchical linear modeling (HLM) analysis provided a methodologically advanced approach for this issue. 
ICT use-related variables are likely to be hierarchical data structures because student ICT use is influenced by 
the characteristics and atmosphere of classes, which are influenced by the school's characteristics. An analysis of 
data that considers this multi-level structure enables us to identify which variables have significant effects. Thus, 
the purpose of this study was to evaluate the exclusive impacts of ICT use on mathematic achievements by 
conducting a hierarchical linear modeling (HLM) analysis that considered variables at both the student- and 
school- levels. The research questions generated were as follows: 

l.How much impact does the use of ICT have on mathematic achievements at the elementary, middle, and 
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high school levels? Specifically, which of the ICT use variables are significantly related to mathematic 
achievements? 

2. How much impact do the background and process variables have on mathematic achievements at the 
elementary, middle, and high school levels? Specifically, which of those factors have a significant 
relationship with mathematic achievements? 

3. According to the different school levels, are there any differences in the impacts of ICT use and the 
related background and process variables on mathematic achievements? 

The Impacts of ICT Use on Learning Achievement 
The Effectiveness of ICT Use 

Since Kulik (1999)’s meta analysis that reported the positive effects of ICT use, many studies have been 
conducted to examine the impacts of ICT use. Fuchs and Wosesman (2004) reported a positive relationship 
between ICT use and academic achievement based on analysis of data from Programmed for International 
Student Assessment (PISA). With the rapid application of new technology in education, ICT allows students to 
explore information from nonlinear sequences, cross-explore new information, improve understanding with the 
aid of visualization tools, and study at any time and at any place. However, despite the positive dimensions, 
some studies criticize ICT use, arguing that it does not play a critical role in improving students’ academic 
achievements. According to Leuven (2004), there is no clear evidence showing that ICT use positively affects 
students' learning outcomes. Correlations between ICT use and academic achievement become significantly 
lower when students’ backgrounds and instruction process-related variables are considered. 

ICT Use Variables 

One of the important variables that influence ICT effectiveness is how often teachers and students use ICT in 
classrooms. The impacts of ICT technology use are mostly measured by the frequency of use in classrooms or at 
home. Given the increased media use in classrooms, it is also important to analyze the ICT teaching and learning 
support system. Another variable to be considered is whether students possess the essential capabilities required 
for ICT technologies (Newhouse, 2002), known as ICT literacy. These include the ability to search for necessary 
information, to attain information by accessing the appropriate places, and to solve problems by processing and 
utilizing information effectively with proper ethics (Kim et al., 2003). It includes the four main components of 
information collection, information processing, information communication, and information use ethics. 

Background and process variables 

There have been debates about whether ICT use could have positive impacts on student achievements. Because 
various variables influence ICT use, the impacts of ICT should be examined, not only considering the ICT use 
variables that are directly related to students' learning outcomes, but also taking into account the background and 
process variables that may indirectly influence achievement. Given that ICT use in classrooms is influenced by 
student characteristics and school support, the background and process variables can be categorized into different 
levels, namely the student-level and the school-level (Kang et al., 2009; Kim et al., 2010; Selim, 2007; Youssef 
& Dahmani, 2008). 

Student-level. Student-level variables include student background and student process factors. For the former, 
research has analyzed parents’ social-economic status, rapport with parents, and parents’ expectation for 
education (Kim, Song, & Huh, 2010). In this study, we also factored in ICT literacy, parents' education levels, 
private tutoring expenditures, and parents' educational support. The factors for the second variable, the student 
process, included elements related to students’ learning processes: self-efficacy, motivation, self-regulated skill 
including learning strategies, and study time (Kim et al., 2010). We also included student attitude and studying 
time. 

School-level School-level variables consist of the school background and school process factors. The average 
teacher's education level and income for each school were used as the school background variables in this study. 
Social-economic background, school location, and school type were not included in this study because the data 
were collected from a specific location, Seoul city, Korea. School process includes a variety of variables that are 
related to school management in delivering ICT. Teachers’ efforts to improve education and the principal’s 
leadership were included in this study because they emphasize the efforts to improve the quality of education 
through ICT utilization (Hermans, et al., 2008). 

METHODS 

Data 

This study used data collected from a 2011 Seoul Education Longitudinal Study. Survey data were collected 
from 5200 fourth grade students at 108 elementary schools, 4600 first grade students at 74 middle schools, and 
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6600 first grade students at 83 high schools. Due to missing data, an analysis was conducted using 5009 
elementary students, 4544 middle school students, and 5240 high school students. Data were also collected from 
1738 teachers who taught fourth grade at various elementary schools, seventh grade at various middle schools, 
and tenth grade at various high schools. 

Research Models 

The HLM approach was employed to examine the distinctive impacts of ICT use. ICT use-related variables were 
categorized into two levels: student-level as level 1 (ICT use, student background variables, and student process 
variables) and school-level as level 2 (ICT use, school background, and school process variables). The ICT use 
variables were entered first, followed by the background and process variables. Table 1 shows the research 
models. 


Table 1: Research Models 


Model 

Input variables 

Basic model 

Mathematic achievements (dependent variables) 

Research model 1 

ICT use variables (students, school) 

Research model 2 

Research model 1 + student background, student processes, school background, 
school processes 


Grand-mean centering was conducted to clearly interpret the interception for all variables, except the gender 
variables, in each model. The equations for level 1 and level 2 in the research model 2 were as follows: 

Level 1 Equation 

Yy = / 3 qj + ftjj x (Frequency : information collection) + @ 2 j x (Frequency : information processing) 

+ ft3j x (Frequency : information communication) + x (Literacy : information collection) 

+ fljj x (Literacy : information processing) + x (Literacy : information communication) 

+ fijj x (EBS viewing hours) + f$gj x (gender) + figj x (parents education) + Pjqj x (tutoring expenditure) 

+ f j jj x (learning attitude) + ftj2j x (study time) + fij ^ x (self efficacy) + ry, 

rj ~N(0, a 2 ) 

Level 2 Equation 

pQj = 7 qo + Yoi x (online support) + yQ2 x (ICT use) + y^j x (EBS use frequency) 

+ yQj x (average teacher education level) + y^j x (family income) + yn^ x (improve teaching) 

+ yQy x (principal's leadership) + ygg x (teachers' creative teaching style) + uqj 

u 0j ~ N <°’ T oo )• 

Pqi = y q o (<1 = V--H) 


Survey instrument 

ICT use frequency and ICT literacy skills were adapted from the computer literacy survey at the OECD 
Programme for International Student Assessment (Park et al., 2010). ICT literacy included three categories: a) 
information collection (program search, CD creation, file transfer, Internet search, file downloading), b) 
information processing and creation (editing digital and graphic images, spreadsheet use, creation of power point 
presentation, multimedia, and web pages) and c) information communication (online chatting, file attachment). 
Learning attitude refers to abilities related to study methods used in the learning process. The six items included 
are related to perceptions about student abilities to elaborate, organize, map, and employ effective study 
methods, to reflect their study processes, and to self-evaluate. According to Park et al. (2010), the validity and 
reliability of the survey instrument were validated (alpha = .83). Self-efficacy was measured using six items of 
three sub constructs (alpha = .86): intrinsic motivation toward learning, self-efficacy, and autonomous learning 
(Park et al., 2010). Weekly study time was measured by summing the regular class time, the afterschool 
participation time, the private tutoring time, and the self-study time for mathematics. 
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Online teaching and learning support were examined by calculating the use of education-related web sites. The 
monthly numbers for using online learning materials and video materials were also examined. 

Average family income was collected, and Average teacher education level was measured. Teachers ’ education 
improvement efforts were measured by summing the annual participation numbers for class demonstrations, peer 
tutors teaching observations, teaching improvement committees, teaching consulting clinic services, and teacher 
community participation. Teachers’ creative teaching effort was measured with two items that asked teachers’ 
perception about the teaching environment, whether it was a good atmosphere for student creativity or an 
unpleasant atmosphere. Principal leadership was measured using three items that examined teachers’ 
perceptions about whether principals were willing to share the school's visions with the teacher, allow teachers to 
participate in decision making processes, and properly conduct teacher evaluations. 

Mathematic achievement was measured using data from the 2010 National Test Scores. 

RESULTS 

The explanations of ICT effect on mathematic achievements for elementary schools, middle schools, and high 
schools are presented in Tables 2, 3, and 4, respectively. 

Table 2: The impacts of ICT on mathematics achievements at elementary schools 

Fixed effects Basic model Research model 1 Research model 2 


Initial value 73.15(0.61)" 73.56(0.58)’ 77.33 (0.65)* 


Student level ICT 



Information collection 

-0.72 (0.20)* 

0.07 (0.33) 

ICT Use Information processing 

0.31(0.16) 

-0.84 (0.26)* 

Frequency Information 

-0.94 (0.21)* 

-0.66 (0.34) 

communication 

Information collection 

1.20 (0.18)* 

1.00 (0.35)* 

ICT Information processing 

-0.21 (0.23) 

-0.27 (0.37) 

Literacy Information 

0.14(0.22) 

0.11 (0.35)* 

communication 

Student (Student background) 

EBS viewing hours 


-0.71 (0.20) 

Gender 


-1.59(0.96) 

Parent education level 


0.88(0.16)* 

Private tutoring expenditure 


1.41 (0.62)* 

Parent education support 


0.03 (0.12) 

Student (Student process) 

Learning attitude 


1.44 (0.16)* 

Hourly average study time 


0.10(0.12) 

Self-directed learning attitude 


-0.05 (0.06) 


School (ICT Use) 


Online teaching-learning 
support system use 

Computer Internet use 


-0.11 (0.12) 

0.17(0.22) 

-0.16(0.10) 

-0.03 (0.14) 

School (School background) 




EBS use frequency 

Average teacher education 
level 

Average family income 



1.19(0.35)* 
-2.28 (0.80)* 
8.10(1.86)* 

School(School process) 




Teaching improvement efforts 
Principals leadership 

Teachers creative teaching 
efforts 



-0.01 (0.07) 
-0.12(0.13) 

0.21 (0.28) 

Random effect(Per.) 

Student level 

316.26(90.88) 

305.03 (91.64) 

227.85 (98.91) 

School level 

31.73 (9.12) 

27.84(8.36) 

2.52(1.09) 

Additional explanation per.(%) 
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3.55 25.30 

_ 12,26 _ 90.95 

"><• 05 


Student level 
School level 


Table 3: The impacts of ICT on mathematics achievements at middle schools 


Fixed effects 

Basic model 

Research model 1 

Research model 

2 

Initial value 

55.34(1.04)* 

55.53 (0.99)’ 

60.80 (0.84)* 

Student level ICT 

Information collection 


-0.88 (0.20)* 

-0.44 (0.38) 

ICT Use Information processing 


0.19(0.14) 

-0.56 (0.18)* 

Frequency Information 

communication 


-2.21 (0.27)* 

-1.51 (0.49)* 

Information collection 


0.23 (0.19) 

0.06 (0.34) 

ICT Information processing 


0.52(0.16)* 

0.30(0.21) 

Literacy Information 

communication 


1.45 (0.31)* 

1.30(0.56)* 

Student (Student background) 

EBS viewing hours 



-0.90 (0.26)* 

Gender 



-0.63 (1.34) 

Parent education level 



1.21 (0.26)* 

Private tutoring expenditure 



3.68 (0.97)* 

Parent education support 



0.07(0.15) 

Student (Student process) 

Learning attitude 



1.17(0.22)* 

Flourly average study time 



0.65 (0.13)* 

Self-directed learning attitude 



0.14(0.10) 

School (ICT Use) 

Online teaching-learning 
support system use 


0.03 (0.32) 

0.20(0.23) 

Computer Internet use 


0.46(0.39) 

0.32(0.26) 

School(School background) 

EBS use 



-0.27 (0.28) 

Average teacher education 
level 



-0.35 (1.35) 

Average family income 



7.88(2.01)* 

School(School process) 

Teaching improvement efforts 



0.60 (0.23)* 

Principals leadership 



0.80 (0.17)* 

Teachers creative teaching 
efforts 



-1.64 (0.38)* 

Random effect(Per.) 

484.0 (88.77 
4) 



Student level 

458.01 (89.32) 

359.49(99.91) 

School level 

61.25< 1L23 

54.78(10.68) 

0.32(0.09) 

Additional explanation per.(%) 
Student level 

5.38 

21.51 


School level 

* 


10.56 

99.42 
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Table 4: The impacts of ICT on mathematics achievements at high schools 


Fixed effects 

Basic model 

Research model 1 

Research model 2 

Initial value 

49.96(1.74)* 

50.08(1.56)* 

52.10(1.05)’ 

Student level ICT 

Information collection 


-1.18(0.26)* 

-0.33 (0.42) 

ICT Use Information processing 


0.76(0.13)* 

0.30 (0.22) 

Frequency Information 

communication 


-2.58 (0.26)* 

-2.10(0.40)* 

Information collection 


-0.06 (0.22) 

0.19(0.44) 

ICT Information processing 


0.40 (0.13)* 

0.04 (0.21) 

Literacy Information 

communication 


1.74 (0.40)* 

1.03 (0.73) 

Student (Student background) 

EBS viewing hours 



-0.99 (0.25)* 

Gender (Girl 



2.11 (1.43) 

Parent education level 



1.06(0.24)* 

Private tutoring expenditure 



2.38(0.93)* 

Parent education support 



-0.11 (0.13) 

Student (Student process) 

Learning attitude 



1.33(0.17)* 

Hourly average study time 



0.49(0.14)* 

Self-directed learning attitude 



0.24 (0.10)* 

School (ICT Use) 

Online teaching-learning 
support system use 


-1.27(0.78) 

0.04 (0.50) 

Computer Internet use 


0.66(0.78) 

-0.36(0.33) 

School (School background) 

EBS use 



-0.99(0.92) 

Average teacher education 
level 



3.53 (1.75)* 

Average family income 



23.20 (4.10)* 

School(School process) 

Teaching improvement efforts 



0.58(0.36)’ 

Principals leadership 



0.23 (0.41) 

Teachers creative teaching 
efforts 



-0.24 (0.80) 

Random effect(Per.) 

m 'l (66.44) 



Student level 

471.03 (69.96) 

393.09(92.06) 

School level 

247A l (33.56) 

202.25 (30.04) 

33.90 (7.94) 

Additional explanation per.(%) 
Student level 


3.83 

16.55 

School level 


18.25 

83.24 


The impact of ICT use on achievements 

The impacts of ICT use and students ICT literacy skills on achievements were measured by the portion of 
variance that could be explained by ICT use among the overall achievement variance. The results showed that 
ICT use explained 3.55 % at the elementary school level, 5.38% at the middle school level, and 3.83% at the 
high school level. 

At the elementary school level, the basic model analysis results showed that the student level variance accounted 
for 90.88 % and the school level variance accounted for 9.12%. As shown in Table 2, when ICT use variables 
were included, research model 2 explained 3.55 % more variance at the student level and 12.56 % more at the 
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school level than did the basic model. At the middle school level, the basic model analysis results showed that 
the student level variance accounted for 88.77% and a school-level variance accounted for 11.23%. 

As shown in Table 3, when the ICT use variables were taken into account, research model 2 explained 5.38 % 
more variance at the student-level than did the basic model. Research Model 2 explained 10.56% more at the 
school-level than did the basic model. 

Finally, as shown in Table 4 for high schools, the basic model analysis results showed that a student-level 
variance accounted for 66.44% and a school-level variance accounted for 33.56%. With the ICT use variables, 
research model 2 explained 3.83 % more variance at the student-level and 18.25% more at the school-level than 
did the basic model. This seems to indicate that the influence of ICT use is three times higher at the middle 
school level. Thus, the impact of ICU use on mathematic achievements in high schools appears to be greater than 
at the middle school or elementary school level. When student-level and school-level were considered together, 
the ICT use variables explained 4.34% of the variance at the elementary school level, 5.96% at the middle school 
level, and 8.67% at the high school level. 

ICT use variables and achievements 

In the elementary schools, the ICT use frequency for information processing and ICT literacy skills for 
information collection and communication had significant impacts on mathematic achievements at the 
student-level. The better were ICT literacy skills for information collection and communication, the higher were 
the academic achievements. When students experienced less frequent information processing, they tended to 
earn higher academic achievements. There was no impact of ICT use at the school-level. 

In the middle schools, ICT use frequencies for infonnation processing and communication and ICT literacy 
skills in information communication had significant impacts on mathematic achievements at the student-level. 
When students had higher ICT literacy skills for infonnation communication, they tended to earn higher scores 
regarding academic achievements. When students experienced less frequent information processing and 
information communication, they tended to earn higher academic achievements. Again, there was no impact of 
ICT use at the school-level. 

In the high schools, as shown at Table 4, the frequent use of ICT for information communication had a 
significant impact on mathematic achievements at the student-level. ICT literacy skills, however, did not have 
any significant impact. The less the students used ICT for information communication, the higher were the 
academic achievements. When the students had greater ICT literacy skills, even though the impacts were not 
statistically significant, they seemed to have higher academic achievements. 

The impacts of background and process variables on achievements 

Our research model 1 analysis results showed that ICT use variables explained 12.26% of the variance in 
mathematic achievements at the elementary school level, 10.56% at the middle school level, and 18.25% at the 
high school level. As shown at Table 2, when background and process variables are included, as in research 
model 2, the variance of math variables was 25.30 % greater at the student-level and 90.95% greater at the 
school-level in elementary schools. As shown in Table 3, when model 2 was applied, the other variables 
explained a 21.51 % greater variance at the student-level and 99.42% greater at the school-level in middle 
school. Furthermore, when the variables of student process, student background, school process, and school 
background were taken into account, the math variance was 16.55 % greater at the student-level and 83.24% 
greater at the school-level in the high schools (Table 4). When the student-level and school-level variances were 
collectively taken into account, the additional independent variables explained 30.79 % more variance at the 
elementary school level, 29.83% more at the middle school level, and 36.58% more at the high school level. 

Background and process variables and achievements 

A variety of background and process variables at both the student-level and school-level seemed to influence 
mathematic achievements. Our analysis results showed that elementary school students tended to show high 
academic achievements when they had higher ICT literacy skills in information collection and information 
communication, more private tutoring tuition, a more positive learning attitude, more e-leaming materials at 
schools, and higher family income (Table 2). On the other hand, they tended to earn lower academic 
achievements when they used more ICT for information processing and when their teachers had more education. 
Next, middle school students tended to show high academic achievements when they less frequently used ICT 
for information processing and communication, had higher ICT literacy skills in information communication, 
spent less hours studying e-leaming materials from EBS, had parents with higher education levels, had more 
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private tutoring tuition, had a more positive learning attitude, had higher family incomes, had leadership from 
the principal, and had less creative teaching efforts (Table 3). 

Lastly, high school students tended to earn higher academic achievements when they used less ICT for 
information communication, spent fewer hours studying e-leaming materials, had a more positive learning 
attitude, had more weekly study hours, had higher self-directed learning attitudes, had teachers with higher 
education levels, had higher family incomes, and had more teacher efforts to improve teaching (Table 4). 

In addition, four variables were significant for the achievements at all school levels: 1) parent education level, 2) 
private tutoring expenditure, 3) learning attitude, and 4) family income. In addition, teachers’ efforts for 
education improvement were also significant as a school-process variable in both middle and high schools. 
Principal’s leadership was only significant at the middle school level. 

DISCUSSION 

ICT is being widely used in all school-levels because it enables students to engage in self-directed learning by 
providing individualized content, overcoming some limitations of traditional teaching methods. This study 
conducted an HLM analysis to rigorously examine the exclusive impacts of ICT use on mathematic 
achievements. 

This study examined the exclusive impacts of ICT by identifying the variance explained by ICT use among 
overall achievements. The exclusive explained variance of ICT on mathematic achievements was 4.34% at the 
elementary school level, 5.96% for the middle school level, and 8.68% at the high school level. A previous study 
on school impacts reported that the explained variance of ICT use was 29% at the middle school level and 36% 
at the high school level when combining all variables both at the student-level and school-level (Kim et al., 
2010). However, the previous study is limited in that it did not separate ICT use from other background and 
process variables. In fact, Kim et al. (2010) incorporated ICT use variables into either weekly study time 
variables in the student-level or resources in the school-level; thus, it was not possible to differentiate their 
distinctive effects. The present study shows that ICT use has distinctive impacts on achievement. In addition, 
when measuring the ICT effectiveness at two-levels, the explained variance of ICT use on achievement in terms 
of school-level was higher than for the student-level. This implies that school-level support is more important 
than students' ICT-related characteristics when using ICT. School-level support is essential in increasing ICT 
learning impacts, such as teachers’ efforts to improve education and principals' leadership. 

The results showed that the use of ICT technologies for communication had negative impacts on achievement. 
This implies that using excessive email, chatting, and blogs can decrease mathematic achievements, and thus, 
these ICT technologies should be used carefully in education. In terms of ICT literacy, students’ abilities in 
information processing have significant impact on achievements. Students need to develop a higher cognitive 
ability to analyze and process information rather than an ability to merely collect and communicate information. 
Future ICT literacy education should emphasize information analysis and information processing. 

Interestingly, ICT self-efficacy explained more portions in the overall variance of mathematic achievement than 
did the frequency of ICT use. These results are consistent with a previous study in the impacts of ICT use, which 
demonstrated that ICT use itself did not affect academic achievement at schools (Aristovnik, 2012; Cox & 
Marshall, 2007; Shaikh, & Khoja, 2011). The ICT use itself is not a proximal variable that affect student 
performance such as their abilities or teaching-learning materials (Wittwer & Senkbeil, 2008). Thus, future ICT 
literacy education should take into account the incorporation of affective elements that help students to increase 
their self-confidence with suitable teaching and learning strategies. 

Lastly, ICT explains more of the overall variance in achievement in high schools than in middle- or elementary 
schools. High school students might use more ICT, such as e-leaming or online-tutoring and, as a result, benefit 
more from using ICT than middle school or elementary school students. Given that ICT is used differently 
according to school-levels, different educational policies are needed to suitably increase the effectiveness of ICT 
at each school level. 
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